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1.  Summary 

The  overarching  objective  of  this  effort  is  to  provide  a  foundation  for  an  affordable, 
ultra-dense,  low-power  computer  system  for  processing  spatio-temporal  data  on  the  fly. 
This  includes  construction  of  a  mixed  mode  (including  analog  and  digital  circuits) 
neuromorphic  computing  system  built  for  rapid  configuration,  dynamic  adaptation,  low- 
power  operation,  and  that  is  well  suited  for  processing  spatio-temporal  data.  Neuromorphic 
or  neuro-inspired  computer  architectures  are  particularly  worthwhile  given  the  increasing 
number  of  big  data  problems  requiring  techniques  and  systems  that  can  capture  knowledge 
from  an  abundance  of  data.  Thus,  the  proposed  memristor-based  dynamic  adaptive  neural 
network  array  (mrDANNA)  addresses  contemporary  application  challenges  while  also 
enabling  continued  performance  scaling. 
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2.  Background 

The  Resistive  Random  Access  Memory  (ReRAM,  aka  -  memristor)  is  a  novel 
form  of  non-volatile  memory  expected  to  replace  a  variety  of  current  memory 
technologies  and  enabling  the  design  of  new  circuit  architectures.  Investigations  of 
ReRAM  as  a  storage  technology  have  shown  a  combination  of  high  storage  density  with 
fast  access  and  write  speeds  [1,2].  In  addition,  the  implementation  of  new  circuit 
architectures,  in  particular  encryption  technologies  into  hardware  components,  has 
attracted  much  attention.  Recently,  the  endurance  and  reliability  of  ReRAM  cells  have 
reached  a  level  at  which  they  are  competing  with  commercially  available  Flash  memory 
and  CMOS  technologies,  making  ReRAM  a  viable  candidate  for  data  storage  and  novel 
logic  and  security  architectures.  To  this  end,  we  have  demonstrated  a  vertically- 
integrated  process  flow  for  fabrication  of  hybrid  CMOS  logic  and  ReRAM. 

Our  memri stive  Dynamic  Adaptive  Neural  Network  Array  (mrDANNA)  is  based 
on  the  Neuroscience-Inspired  Dynamic  Architecture  (NIDA)  [1-5],  developed  by 
researchers  at  the  University  of  Tennessee,  Knoxville  (UTK)  as  an  approach  to  applying 
neuromorphic  principles  to  a  wide  variety  of  applications.  Key  features  of  the  NIDA 
architecture  include:  1)  a  spiky  representation  of  data,  2)  the  ability  for  the  system  to  adapt 
during  run-time,  and  3)  a  synaptic  representation  including  delay  distance  as  well  as  weight 
information.  The  inclusion  of  delay  distance  (i.e.  a  programmable  delay  between  pre-  and 
post-synaptic  neurons)  is  expected  to  be  of  particular  benefit  in  the  processing  of  spatio- 
temporal  data.  The  structure  and  simplicity  of  the  NIDA  architectural  model  has  recently 
been  leveraged  in  the  development  of  a  Dynamic  Adaptive  Neural  Network  Array 
(DANNA)  [6],  an  efficient  digital  system  constructed  from  a  basic  element  that  can  be 
configured  to  represent  either  a  neuron  or  a  synapse.  Unique  characteristics  of  the 
NIDA/DANNA  approach  over  other  neuromorphic  or  neuroscience-inspired  systems 
include:  a  simplified  neuron  model,  a  higher  functionality  synapse  model,  real-time 
dynamic  adaptability,  configurability  for  the  overall  neuromorphic  structure  (e.g.  number 
of  neurons,  number  of  synapses  and  connections),  and  scalability  for  element  performance 
and  system  capacity. 
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3.  Results  and  Discussion 

An  Evolutionary  Optimization  (EO)  environment  has  been  developed  at  UTK  to 
configure  the  neural  networks  in  a  DANNA  [1-6].  The  EO  trains  over  parameters  of  the 
network  (weights  and  delay  distances  on  synapses  and  thresholds  on  neurons)  as  well  as 
the  structure  (the  number  and  placement  of  neurons  and  synapses)  and  the  dynamics  of  the 
network.  The  dynamics  of  the  network  are  directly  embedded  in  the  structure  itself  (delays 
in  the  synapse  and  charges  in  the  neuron).  Most  other  artificial  neural  network 
implementations  have  a  predefined,  fixed  structure  rather  than  one  detennined  by  a 
dynamic  optimization  method  as  in  our  approach. 

Memristive  synapses  have  been  interfaced  with  CMOS  neurons  in  [7-8]  and  a 
cellular  non-linear  network  based  on  memristors  was  proposed  in  [9].  Memristors  were 
used  to  build  non-volatile  two-level  and  multi-level  memories  in  [10]  and  [11-13], 
respectively.  Memristors  can  also  be  used  in  digital  logic  as  programmable  switches  in 
switching  blocks  [14]  and  to  build  block  memories.  To  implement  the  proposed 
mrDANNA  test  chip,  we  are  using  a  hybrid  CMOS/Memristor  process  that  we  have 
developed  at  CNSE  /  SUNY  Polytechnic  Institute.  The  process  integrates  metal-oxide 
memristors  in  the  metal  layers  of  the  65  mn  lOLPe  CMOS  process  from  IBM,  leading  to 
a  seamless  CMOS/memristor  integration  process.  The  seamless  integration  of  CMOS 
with  memristive  technology  is  a  unique  feature  as  compared  to  related  efforts  where 
memristive  devices  are  integrated  post-fabrication  on  an  existing  CMOS  chip.  The 
current  state  of  device  development  shows  highly  reliable  devices  performing  with  an 
endurance  of  over  450k  cycles  in  a  two  resistive  state  switching  mode.  An  average  LRS 
and  HRS  of  7kf2  and  40kQ,  respectively,  were  observed  during  pulsing  measurements. 
The  on/off  ratio  and  LRS  are  likely  to  increase,  which  is  critical  for  low  power  operation, 
by  manipulating  the  thicknesses  and  stoichiometries  in  the  ReRAM  device  film  stack. 

The  devices  show  an  excellent  readout  stress  insensitivity,  with  the  resistive  states  being 
unsusceptible  to  trillions  of  nanosecond  pulse  readouts.  In  addition,  a  low  positive 
temperature  dependence  (5.9e-4  1/C)  results  in  little  change  to  the  circuit  perfonnance 
over  a  large  range  of  temperatures.  Recent  results  also  indicate  capability  for  controllable 
analog/multi-level  switching  in  our  memristive  devices,  which  is  key  for  their  application 
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as  synapses  in  the  mrDANNA.  This  work  briefly  illustrates  our  technological  capabilities 
and  design  approach  towards  the  implementation  of  the  final  mrDANNA  prototype. 


Figure  1.  Simplified  view  of  two-stage  mrDANNA  including  several  pre-synaptic  neurons  driving 
a  single  post-synaptic  neuron  through  memristor  based  synapses.  The  local  feedback  loop  in  the 
neuron  enables  dynamic  adaptation  where  memristor  weights  are  updated  during  run  time. 


Fig.  2.  Schematic  of  memristor  devices  fabricated  at  CNSE,  showing  tungsten  (W)  bottom  contacts 
for  integration  with  CMOS  (left).  Consistent  switching  parameters  have  been  achieved  with  these 
devices,  including  reproducible  LRS  resistance  for  >4xlOA5  cycles  (right). 
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